Bio-Inspired Computer Vision: Towards a Synergistic Approach of Artificial and Biological Vision

To appear in CVIUStudies in biological vision have always been a great source of inspiration for design of computer vision algorithms. In the past, several successful methods were designed with varying degrees of correspondence with biological vision studies, ranging from purely functional inspiration to methods that utilise models that were primarily developed for explaining biological observations. Even though it seems well recognised that computational models of biological vision can help in design of computer vision algorithms, it is a non-trivial exercise for a computer vision researcher to mine relevant information from biological vision literature as very few studies in biology are organised at a task level. In this paper we aim to bridge this gap by providing a computer vision task centric presentation of models primarily originating in biological vision studies. Not only do we revisit some of the main features of biological vision and discuss the foundations of existing computational studies modelling biological vision, but also we consider three classical computer vision tasks from a biological perspective: image sensing, segmentation and optical flow. Using this task-centric approach, we discuss well-known biological functional principles and compare them with approaches taken by computer vision. Based on this comparative analysis of computer and biological vision, we present some recent models in biological vision and highlight a few models that we think are promising for future investigations in computer vision. To this extent, this paper provides new insights and a starting point for investigators interested in the design of biology-based computer vision algorithms and pave a way for much needed interaction between the two communities leading to the development of synergistic models of artificial and biological vision

Understanding the impact of recurrent interactions on population tuning: Application to MT cells characterization

International audienceA ring network model under neural fields formalism with a structured input is studied. Bifurcation analysis is applied to understand the behaviour of the network model under different connectivity regimes and input conditions. The parameter regimes over which the localised input bumps could be preserved, combined or selected are used to identify the potential network regimes under which direction selective cells in MT area exhibiting analogous behaviour could be operating. The parameter regimes are further explored to identify possible transitions in the tuning behaviour with respect to change of driving stimuli as observed in experimental recordings

Improving FREAK Descriptor for Image Classification

International audienceIn this paper we propose a new set of bio-inspired descrip- tors for image classification based on low-level processing performed by the retina. Taking as a starting point a descriptor called FREAK (Fast Retina Keypoint), we further extend it mimicking the center-surround organization of ganglion receptive fields.To test our approach we com- pared the performance of the original FREAK and our proposal on the 15 scene categories database. The results show that our approach out- performs the original FREAK for the scene classification task

Improving FREAK Descriptor for Image Classification

International audienceIn this paper we propose a new set of bio-inspired descrip- tors for image classification based on low-level processing performed by the retina. Taking as a starting point a descriptor called FREAK (Fast Retina Keypoint), we further extend it mimicking the center-surround organization of ganglion receptive fields.To test our approach we com- pared the performance of the original FREAK and our proposal on the 15 scene categories database. The results show that our approach out- performs the original FREAK for the scene classification task

What can we expect from a V1-MT feedforward architecture for optical flow estimation?

Motion estimation has been studied extensively in neuroscience in the last two decades. Even though there has been some early interaction between the biological and computer vision communities at a modelling level, comparatively little work has been done on the examination or extension of the biological models in terms of their engineering efficacy on modern optical flow estimation datasets. An essential contribution of this paper is to show how a neural model can be enriched to deal with real sequences. We start from a classical V1-MT feedforward architecture. We model V1 cells by motion energy (based on spatio-temporal filtering), and MT pattern cells (by pooling V1 cell responses). The efficacy of this architecture and its inherent limitations in the case of real videos are not known. To answer this question, we propose a velocity space sampling of MT neurons (using a decoding scheme to obtain the local velocity from their activity) coupled with a multi-scale approach. After this, we explore the performance of our model on the Middlebury dataset. To the best of our knowledge, this is the only neural model in this dataset. The results are promising and suggest several possible improvements, in particular to better deal with discontinuities. Overall, this work provides a baseline for future developments of bio-inspired scalable computer vision algorithms and the code is publicly available to encourage research in this direction

Decoding MT Motion Response for Optical Flow Estimation: An Experimental Evaluation

Published in the 23rd European Signal Processing Conference (EUSIPCO)Motion processing in primates is an intensely studied problem in visual neurosciences and after more than two decades of research, representation of motion in terms of motion energies computed by V1-MT feedforward interactions remains a strong hypothesis. Thus, decoding the motion energies is of natural interest for developing biologically inspired computer vision algorithms for dense optical flow estimation. Here, we address this problem by evaluating four strategies for motion decoding: intersection of constraints, maximum likelihood, linear regression on MT responses and neural network based regression using multi scale-features. We characterize the performances and the current limitations of the different strategies, in terms of recovering dense flow estimation using Middlebury benchmark dataset widely used in computer vision, and we highlight key aspects for future developments

A retinotopic neural fields model of perceptual switching in 2D motion integration

International audienc

Adaptive Motion Pooling and Diffusion for Optical Flow

We study the impact of local context of an image (contrast and 2D structure) on spatial motion integration by MT neurons. To do so, we revisited the seminal work by Heeger and Simoncelli (HS) using spatio-temporal filters to estimate optical flow from V1-MT feedforward interactions. However, the HS model cannot deal with several problems encountered in real scenes (e.g., blank wall problem and motion discontinuities). Here, we propose to extend the HS model with adaptive processing by focussing on the role of local context indicative of the local velocity estimates reliability. We set a network structure representative of V1, V2 and MT. We incorporate three functional principles observed in primate visual system: contrast adaptation, adaptive afferent pooling and MT diffusion that are adaptive dependent upon the 2D image structure (Adaptive Motion Pooling and Diffusion, AMPD).We evaluated both HS and AMPD models performance on Middlebury optical flow estimation dataset. Our results show that the AMPD model performs better than the HS model and its overall performance is comparable with many modern computer vision. The AMPD model has to be further improved by integrating feedback from MT to V1 to better recover true velocities around motion discontinuities. However, we think that this adaptive model can serve as a ground for future research in biologically-inspired computer vision